Knowledge Transfer in Artificial Learning

This document presents the main research problems addressed during my PhD studies. All these researches are led inside the two teams DBWeb in Télécom ParisTech and LInK (Learning and Integration of Knowledge) in AgroParisTech, both located in Paris, and supervised by Pr. Jean-Louis Dessalles and Pr. Antoine Cornuéjols. My researches focus on learning theory both in the perspective of symbolic machine learning and of learning in continuous domains. I aim at finding an information-theoretic principle guiding information transfer in learning. The start point of these researches is the idea that most machine learning takes a strong stationary hypothesis for granted. The general framework of statistical learning (mainly supervised and semi-supervised learning) considers two data sets: a learning data set (from which the concepts have to be learned) and a test data set (on which quality of the learned concepts is evaluated). The key idea of current learning methods and theories is to assume that training data and test data are independent and identically distributed (i.i.d.). However this strong hypothesis does not hold in many cases: either the data generation process evolves over time (aging effect, trending effect...) or the data belong to a different domain. Because similar questions of transfer and domain adaptation had already been addressed in analogical reasoning, we proposed to use an approach based on Kolmogorov complexity instead of probabilities. Kolmogorov complexity is a measure of the information contained inside an object. The use of Kolmogorov complexity in machine learning is accepted by the community, but mainly in a stationary point of view (when the key concept does not vary); we proposed to extend its use to non-stationary environments, in the same way as done in analogical reasoning. A presentation of these issues is given in section 2. The strong similarity between transfer learning and analogical reasoning led me to consider this issue in my researches. Analogical reasoning consists in situations of the form “‘b’ is to ‘a’ as ‘d’ is to ‘c’”. Because its value has already been demonstrated, I focus on Hofstadter’s micro-world, made up of strings of alphabetical characters that can be described with simple concepts like ‘predecessor’, ‘successor’ or ‘repetition’. The use of Kolmogorov complexity for analogical reasoning had already been considered, but our approach is slightly different. We